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(57) A method and apparatus for modifying an 
image defined by data representing the colour 
component content of each pixel of the image. 
The method comprises, for each colour compo- 
nent of each pixel 

(a) generating unsharp (U) and sharp (S) data 
and performing an unsharp masking algorithm 
on the sharp and unsharp data to generate 
fringe data (F) ; 

(b) modifying the fringe data with a weighting 
function (4>) determined in accordance with the 
values of the sharp or unsharp data for all of the 
colour components (U C ,U M? U Y ) of the pixel ; and 

(c) combining the modified fringe data with 
the pixel data to generate modified pixel data. 

By applying a weighting function related to all 
the unsharp values for the pixel and hence to 
the input colour component content of the 
input signals of a particular sample area, the 
fringe amplitude is varied according to the col- 
our of the region thereby enabling sharpening 
or smoothing to be appropriate to the part of 
the image e.g. sharpening hair and smoothing 
skin colours. 
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This invention relates to a method of modifying a colour image. 

It is normal in the processing of an image or graphic arts reproduction to sharpen edges and enhance fine 
detail by means of unsharp masking (USM). Typically this is performed by a convolution process, passing a 
filter window across the image to take a weighted average of the surrounding pixels at each position, then gen- 
5 erating some function of the so-called 'sharp' (original value of central pixel) and 'unsharp' (weighted average 
of neighbours) signals. 

Figure 7 shows the relationship between the regions covered by the sharp and unsharp signals. The sharp 
signal in this instance is the value of the single pixel 70 at the current processing location. The unsharp signal 
signal is a weighted average of the 25 pixels in a 5 by 5 pixel region 71 centred on the same location. The 

10 weighting function may be a simple average of all pixels, ie. equal contribution from all elements, or may give 
greater contribution to the pixels nearer the centre. The regions sizes used in deriving both sharp and unsharp 
signals may be either smaller or larger. The technique of convolution is well known in digital image processing 
applications, see for example the text "Digital Image Processing (Second Edition) by R.C.Gonzalez and P- 
.Wintz, published in 1987 by Addison- Wesley, pp.161-183. 

15 The principle of unsharp masking is to emphasise high-frequency fringe information in the image as illu- 

strated in Figure 8a. A function of the difference between the original sharp (local) signal S and the low fre- 
quency unsharp (neighbourhood average) signal U is calculated. Figure 8a shows graphs of frequency (f) 
against number of pixels for sharp (S) data, unsharp (U) data and the function kF(S-U) which gives the high 
frequency fringe data. 

20 Figures 8b-8e show the successive stages for addition of a thresholded difference signal (sharp minus 

unsharp) to the original sharp signal. Figure 8b shows the original sharp pixel signal S and the low-pass filtered 
unsharp signal U. The difference (S-U) is shown in Figue 8C, which also shows the threshold values <T,-T). 
The thresholded fringe signal is shown in Figure 8d and the final, enhanced output signal P is shown in Figure 
8e, in which the fringe signal has been combined with the original signal. Typical equations for the process 

25 are: 

P= S + K.f(S - U) 

where 

S = sharp signal (local value) 

U - unsharp signal (neighbourhood average) 

30 f() = suitable fringe generation function 

K = weighting factor for degree of USM applied 

P = output signal (enhanced) 

The process of unsharp masking in image scanners for the graphic arts is described in detail in "Electronic 
Color Separation" by R.K.Molla published in 1988 by R.E.Printing of Charleston WV, pp. 225-240. 

35 Such a method may be applied on the fly in a graphic arts scanner as the image is being scanned, or as 

a post-scan processing operation by the computer on the stored digital image. 

The purpose of unsharp masking is to increase the apparent sharpness of boundaries in the image. In elec- 
tronic image reproduction unsharp masking is achieved by deriving from the picture signals sharp and unsharp 
signals equivalent to viewing a small area, and a large area incorporating the small area of the image, respec- 

40 tively. These signals are then combined in a predetermined manner so that the contrast on either side of the 
boundary is increased. Thus, at a boundary between light and dark areas, the light area adjacent the boundary 
is made lighter and the dark area adjacent the boundary is made darker. This gives the visual effect of increased 
sharpness. 

Another customary and necessary processing operation carried out on an image is to apply colour cor- 
45 rection to compensate for differences in colour gamut between original photographic dyes and printing inks, 
deficiencies in the original such as casts due to lighting, wrong exposure or film processing error and editorial 
colour adjustment Typically, colour correction is performed by a look-up table (LUT) operation in which the 
table is loaded by a colour correction function that is the composite of all the factors. 

It has been shown that an improved image results from parallel, rather than serial operation of USM and 
so colour correction. See European Patent Publication EP-A-0171954. In a typical graphic arts scanner, the in- 
coming digital signals derived from the light transmitted through the red, green and blue filters of the image 
sensing means are converted into sharp colour corrected signals forthe four printing inks cyan, magenta, yellow 
and black. 

However, the problem with this method is that the sharpening of edges and fine detail is applied to all colour 
55 separations. Although the function f(S,U) may be sensitive to contrast differences between the S and U signals, 
where S is a sharp signal and U an unsharp signal, typically by reducing small differences and amplifying large 
differences, the resultant fringe signal f(S,U) is independent of the actual colour of the image. This can result 
in the enhancement of both the wanted edge detail and unwanted noise and grain detail present in the original 
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film. 

Another common operational requirement is to be able to enhance or sharpen some regions of an image 
whilst simultaneously smoothing others. Avery common example is in the reproduction of portrait photographs 
where the hair and eyelashes are made as sharp as possible, whilst the skin is to be made smooth to disguise 
5 blemishes. Typically, this operation must be performed as a subsequent image retouch operation with the use 
of one or more spatial masks to discriminate between the areas to be sharpened and smoothed and cannot 
be included in the scanning process. 

A further conventional image enhancement technique is described in US-A-2962548. 

In accordance with one aspect of the present invention, we provide a method of modifying an image de- 
10 fined by data representing the colour component content of each pixel of the image, the method comprising, 
for each colour component of each pixel, 

(a) generating unsharp (U) and sharp (S) data and performing an unsharp masking algorithm on the sharp 
and unsharp data to generate fringe data (F); 

(b) modifying the fringe data with a weighting function (<j>) determined in accordance with the values of 
15 the sharp or unsharp data for ail of the colour components of the pixel; and 

(c) combining the modified fringe data with the pixel data to generate modified pixel data. 

In accordance with a second aspect of the present invention we provide apparatus for modifying an image 
defined by data representing the colour component content of each pixel of the image, the apparatus com- 
prising generating means to generate unsharp and sharp data, for each colour component of each pixel; proc- 
20 essing means to perform an unsharp masking algorithm on the unsharp and sharp data to generate fringe data, 
means to modify the fringe data with a predetermined weighting function in accordance with the values of the 
sharp or unsharp data for all of the colour components of the pixel; and means for combining the modified fringe 
data with the pixel data to generate modified pixel data. 

By applying a weighting function related to the colour of the region containing the pixel (ie. the colour as 
25 defined by a point in 3D colour space given by the sharp or unsharp data values), the fringe amplitude (ie. 
degree of sharpening) is varied according to the colour of the region. This enables sharpening or smoothing 
to be appropriate to the part of the image e.g. sharpening hair and smoothing skin colours. By using the unsharp 
signal values there is greater resistance to noise in the original pixel data, but sharp signal values could also 
be used. 

30 Typically the unsharp data is determined by calculating a weighted average of the pixel data in a region 

(such as a 5*5 pixel square) surrounding the pixel. Alternatively, the unsharp data may be calculated as some 
other function of the neighbourhood pixel values. 

Typically, the sharp data is the same as the pixel data. However, it may be colour corrected pixel data, or 
some other function of the pixel data. 
35 Preferably the degree of sharpening of the various parts of the image is determined by an operator pointing 

to areas of the image to be smoothed and sharpened respectively. The average colour of each area is then 
calculated, and the value of the weighting function for that colour is set according to the required degree of 
sharpening or smoothing. Alternatively the operator may select a point on the image, and the average colour 
of an area surrounding that point (which may or may not be the same area as the region used to calculate un- 
40 sharp data values) is calculated and used to determine part of the weighting function. Alternatively, the weight- 
ing function may be set automatically according to some predetermined rule set. 

Preferably, the weighting function is a mathematically continuous function in the coordinate space defined 
by the sharp or unsharp colour signals. 

Preferably the pixel data is colour corrected before being combined with the modified fringe data. Alter- 
45 natively the pixel data may not be colour corrected, or may be corrected after sharpening. 

An example of a method and apparatus for colour correction of an image in accordance with the present 
invention will now be described and contrasted with a conventional method and apparatus with reference to 
the accompanying drawings, in which:- 

Figure 1 illustrates diagramaticafly an example of image scanning apparatus; 
so Figure 2 is a block diagram of a conventional USM/ colour correction circuit; 

Figure 3 is a graph of a typical fringe function for use in a method according to the present invention; 
Figure 4 is a block diagram of apparatus for carrying out the method of the present invention; 
Figure 5 is a block diagram of the circuitry for generating a weighting function; 
Figure 6 is a graph of a weighting function; 
55 Figure 7 illustrates how an unsharp and sharp image is obtained; 

Figure 8 shows the successive stages of an unsharp masking process; and, 
Figure 9 shows a more general form of the generator of the weighing function. 

In Figure 1 , a transparent original 1 to be reproduced is wrapped around the surface of a transparent drum 
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1 2. A xenon lamp 3 directs light rays into the drum and on to a 45° mirror 4, from which the rays pass through 
the wall of the drum and through the transparent original 1. These light rays reach an analysing head 5 con- 
taining colour filter and photoelectric devices such that signals representing the red, blue and green densities 
of the scanned element of the picture 1 are produced on lines 6,7 and 8 respectively. The analysing head 5 is 

5 mounted on a lead screw 9 which is driven in synchronism with the rotation of the drum 12 by a motor 10. As 
a consequence, the analysing head sees a point on the drum 12 which, as the drum rotates and the analysing 
head moves along its lead screw, traces out a helical path along the drum 12 and consequently traces out a 
number of parallel scanning lines on the original 1. 

A light-sensitive sheet 11 to be exposed is mounted on the drum 12. The drum 12 is mounted on a shaft 

w 13 driven by a motor 14. The motor also drives a slotted disc 1 5, the slotted periphery of which rotates between 
a light source 1 6 and a photoeletric cell 1 7. Pulses derived from the photoelectric cell 1 7 are applied to a control 
unit 18 which controls the rotation of the motor 10, driving the lead screw for the analysing head, and a motor 
1 9 which drives a lead screw 20 on which is mounted an exposing head 21. The exposing head 21 includes a 
light source which is modulated by a signal on a line 22. This signal is derived from the input signals on lines 

15 6,7 and 8 in the following manner. 

The RGB signals on the lines 6,7 and 8 are first applied to analogue to digital converters 23, the digital 
outputs of which are fed to an enhancement unit 24. The enhancement unit 24 will be described in more detail 
below and as will be explained provides four output signals representing the converted and corrected signals 
for cyan, yellow and magenta, together with a black signal. A channel selector 25 receives the four signals, 

20 representing the cyan, yellow and magenta printer values and the black printer value, and selects the one which 
corresponds to the separation to be made with the light-sensitive sheet 11. This signal is converted into ana- 
logue form in a D/A converter 26 and is then used to modulate the light source in the exposing head 21. 

The enhancement unit 24 in a conventional apparatus is partly shown in Figure 2. Equivalent circuits are 
provided for producing magenta and yellow outputs from green and blue density input signals respectively. 

25 The signal R representing the red density of a scanned element of the picture 1 is input in parallel to a USM 
filter 31 (the output of which is the unsharp signal U), a LUT 32 (as the sharp signal S) and a colour conversion 
circuit 33. In the LUT 32 the unsharp masking algorithm is applied to the sharp and unsharp inputs, S and U. 
Typically, the unsharp masking is performed by a convolution process as described above, in the USM filter 
31. 

30 In its simplest form, the unsharp masking algorithm applied by each fringe generator LUT 32 may be de- 

fined as follows: 

F = k(S - U) 

where k is the slope of the fringe signal F as a linear function of the difference between the sharp signal 
S and unsharp signal U. 

35 In practice it is common to refine the function by applying a threshold function to the difference signal S- 

U, which represents the localised contrast gradient, in order to allow different degrees of enhancement for small 
and large contrast changes: 

40 { \z y {S - U) when |S-U| < T 

F = { 

{ k 2 (S - U) when |S-U| >=T 

45 where 

k 1( k 2 are two slopes for the fringe function and T is a threshold value of the difference signal S-U 

Figure 3 shows a typical fringe function F(S,U) with negative slope k 1 for signal excursions less than the 
threshold, resulting in signal smoothing, and positive slope k 2 for signal excursions greater than the threshold, 
resulting in the enhancement of fringes. 
so The outputs of the colour correction circuit 33 and LUT 32 (fringe signal F) are combined by a mixer 34 

and output at 41 for storage or to drive an expose head. Unsharp masking and colour correction may be applied 
on the fly in a graphic arts scanner as the image is being scanned, or as a post scanning processing operation 
by a computer on the stored digital image. 

The colour conversion circuit 33 typically comprises a look-up table loaded by a colour correction function 
55 which is a composite of all the factors which influence the colour gamut as described above. Signals repre- 
senting the green and blue density of the scanned element of the picture 1 are input to the colour correction 
circuit 33 as well as the signals for the red density. 

By contrast the enhancement unit 24 for use with the present invention is shown in Figure 4. Again this 
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example is for the red density of the signal from the picture 1. Equivalent circuitry for the green and blue den- 
sities is also provided, but not shown. 

The red density signal R is input in parallel to the colour conversion circuit 33, the USM filter 31 and the 
LUT 32. The unsharp signal U c output from the USM filter 31 is then input in parallel to the LUT 32 and an 
5 additional LUT 36. The additional LUT 36 generates a weighting function $ from the unsharp signal U c and the 
other unsharp signals U m , U y whilst the LUT 32 applies the unsharp masking algorithm to the sharp and unsharp 
signals S,U C . 

The output of the LUT 32 is modified by the weighting function $ from the LUT 36 by multiplier 40 so that 
the application of the fringe function of Fig. 3 is dependent on the colour of the pixels of the part of the picture 

10 1 to which it is applied. Figure 6 shows an example of a suitable weighting function <(> (U). The weighting function 
should be a mathematically continuous, smoothly changing function of the three input signals. This example 
results in full fringe at 60, and no fringe at 61. Various methods of generating such functions are possible as 
used in the generation of colour selected spatial masks for image retouching purposes. An example of such 
functions in three-dimensions is described in EP-A-0441558. The corrected signal value from mixer 34 may 

15 then be output to a printer. 

In Figure 5 an example of the circuitry for generating the weighting function 4> (U c , U M , U Y ) is shown in 
more detail. Individual unsharp signals U for cyan, magenta and yellow respectively are input to a splitter 35 
where the least significant bits can be discarded to reduce the memory requirements of the LUT 36 and hence 
reduce the set-up time. Next, these signals are input to the LUT 36 where the weighting function $ (U) is gen- 

20 e rated. 

The preferred option of taking the four most significant bits of each of the three input signals results in a 
12-bit address to the LUT 36 containing 4096 entries. 

The fringe weighting function <J> could also be generated from the original pixel data or from the sharp sig- 
nals S R , S Q , S b , though the preferred option is to use the unsharp signals as shown in Figures 4 and 5, because 
25 it increases the resistance to noise in the pixel data of the original image. A more general form of the generator 
for the weighting function is shown in Figure 9. Instead of the multiplier 40 the fringe signal F and the weighting 
signal <j> are input to the LUT 90, which produces the weighted fringe signal for input to mixer 34. The LUT 90 
can be pre-loaded with any suitable composition function of the two inputs. The mixer 34 could likewise be 
replaced by a LUT. 

30 In practice the operator of the equipment trains the weighting function by pointing to the regions of the 

image to be smoothed and sharpened respectively on a touch sensitive screen 42 (or by some other input de- 
vice). A processor 43 then samples the pixels in each area to determine the average colour, and sets up the 
function in the lookup-table 36 accordingly. 

It should be understood that the invention applies not only to conventional drum scanners as described 

35 but also to other types of scanners, such as those using CCD arrays, and to video cameras, in fact to any type 
of real time image acquisition and conditioning device or process and to any kind of post-scan image process- 
ing operation. 



40 Claims 

1 . A method of modifying an image defined by data representing the colour component content of each pixel 
of the image, the method comprising, for each colour component of each pixel, 

(a) generating unsharp (U) and sharp (S) data and performing an unsharp masking algorithm on the 
45 sharp and unsharp data to generate fringe data (F); 

(b) modifying (40) the fringe data with a weighting function (<M determined in accordance with the values 
of the sharp or unsharp data for all of the colour components (U C .U M ,U Y ) of the pixel; and 

(c) combining the modified fringe data with the pixel data to generate modified pixel data (41). 

so 2. A method according to claim 1 , wherein the unsharp data is generated by calculating a weighted average 
of the pixel data in a region (71) surrounding the pixel (70). 

3. A method according to daim 1 or 2, wherein the weighting function is determined by selecting areas of 
the image having different required degrees of modification, and calculating the weighting function in ac- 

55 cordance with the average colour and required degree of modification of each selected area. 

4. A method according to any of the preceding claims, wherein the weighting function (<(>) is a mathematically 
continuous function in the coordinate space defined by the unsharp colour signals. 
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. A method according to any of the previous claims, further comprising applying colour correction to the 
pixel data, wherein the pixel data in step (c) is colour corrected pixel data. 

. Apparatus (24) for modifying an image defined by data representing the colour component content of each 
pixel of the image, the apparatus comprising generating means (23,31) to generate unsharp (U) and sharp 
(S) data, for each colour component of each pixel; processing means (32) to perform an unsharp masking 
algorithm on the unsharp and sharp data to generate fringe data (F), means (36,40) to modify the fringe 
data with a predetermined weighting function (4>) in accordance with the values of the sharp or unsharp 
data for all of the colour components (U C ,U M ,U Y ) of the pixel; and means (34) for combining the modified 
fringe data with the pixel data to generate modified pixel data (41). 

, Apparatus according to claim 6, wherein the generating means (31) generates unsharp data by calculating 
a weighted average of the pixel data in a region (71) surrounding the pixel (70). 

Apparatus according to claim 6 or 7, comprising means (42) to select areas of the image having different 
required degrees of modification, and means (43) of calculating the weighting function in accordance with 
the average colour and required degree of modification of each selected area. 

. Apparatus according to any of claims 6 to 8, wherein the weighting function (<|>) is a mathematically con- 
tinuous function in the coordinate space defined by the unsharp colour signals. 

0. Apparatus according to any of claims 6 to 9, further comprising a colour converter (33) to apply colour 
correction to the pixel data, wherein the colour corrected pixel data is combined with the modified fringe 
data. 
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Fig. 8a. 
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